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Abstract 


Indirect  photography  is  a  recently  demonstrated  technique  which  expands  on  the 
principles  of  dual  photography  and  allows  for  the  imaging  of  hidden  objects.  A  camera 
and  light  source  are  collocated  with  neither  having  line-of-sight  access  to  the  hidden 
object.  Light  from  the  source,  a  laser,  is  reflected  off  a  visible  non-specular  surface  onto 
the  hidden  object,  where  it  is  reflected  back  to  the  initial  non-specular  surface  and 
collected  by  the  camera.  This  process  may  be  repeated  numerous  times  for  various  laser 
spot  positions  to  yield  slightly  different  camera  images  due  to  a  variation  in  the 
illumination  of  the  object.  These  images  can  then  be  used  to  construct  an  “indirect” 
image  of  the  hidden  object.  This  thesis  provides  an  alternative  method  of  processing  the 
camera  images  by  modeling  the  system  as  a  set  of  transport  and  reflectance  matrices. 
This  approach  reduces  the  required  size  of  the  visible  scattering  surface.  Matrix 
formulation  and  those  parameters  shown  in  simulation  to  improve  indirect  image  quality 
as  measured  by  a  modified  MTF,  including  the  method  of  matrix  inversion,  and  number 
and  pattern  of  laser  spots,  are  discussed. 
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MATRIX  DETERMINATION  OF  REFLECTIVITY 


OF  HIDDEN  OBJECT  VIA  INDIRECT  PHOTOGRAPHY 

1.  Introduction 


1.1  Problem  Statement 

The  United  States  Air  Force  and  other  branches  of  the  United  States  military 
constantly  gather  intelligence  on  unknown  areas,  both  as  a  means  to  identify  potential 
targets  as  well  as  to  conduct  reconnaissance  before  entering  an  area.  While  numerous 
methods  currently  exist  to  accomplish  this,  an  adversary  wishing  to  prevent  friendly 
personnel  from  acquiring  an  image  of  an  area  merely  needs  to  block  the  line  of  sight, 
thereby  limiting  the  amount  of  intelligence  available  to  the  military.  An  ability  to  gather 
images  from  areas  without  a  direct  line  of  sight  would  be  a  great  improvement.  This 
paper  discusses  the  possibilities  and  limits  of  using  indirect  photography,  particularly  in 
using  a  matrix  formulation,  as  a  technique  to  accomplish  this  goal. 

1.2  General  Issue 

Dual  photography  is  a  relatively  recent  photographic  technique  which  takes 
advantage  of  Helmholtz  reciprocity  to  allow  the  positions  of  camera  and  illuminating 
light  source  to  be  interchanged  (Sen  et  al.,  2005).  Dual  photography  takes  advantage  of 
this  to  allow  information  which  is  available  (or  visible)  to  the  light  source,  but  not  the 
camera,  to  be  restored  and  captured  by  the  camera.  While  an  interesting  advance  with 
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great  application  in  computer  generated  graphics,  dual  photography  requires  that  the  light 
source,  in  this  case  a  pixilated  projector,  be  placed  in  a  position  with  a  clear  view  of  the 
object  of  which  an  image  is  desired.  As  a  result,  dual  photography  cannot  be  used  to 
obtain  imagery  which  is  otherwise  unavailable  to  either  the  light  source  or  the  camera. 
Indirect  photography  takes  this  one  step  further.  As  demonstrated  by  Lieutenant  Colonel 
Mark  Hoelscher,  USAF,  using  multiple  non-specular  surfaces,  it  is  possible  to  co-locate 
the  camera  and  light  source  (in  this  case  a  laser),  neither  of  which  has  a  direct  line  of 
sight  to  the  object  in  question,  and  still  obtain  recognizable  visuals  of  that  object,  though 
not  as  clearly  as  can  be  accomplished  via  dual  photography  (Hoelscher,  2011).  In 
indirect  photography,  one  point  on  a  surface  (hereafter  referred  to  as  surface  1)  is 
illuminated,  causing  non- specularly  reflected  light  to  be  cast  upon  the  object  (also 
referred  to  here  as  surface  2)  and  then  back  to  either  the  initial  or  some  other  surface 
(referred  to  here  as  surface  3).  This,  however,  was  accomplished  by  moving  the  field  of 
view  (FOV)  of  the  camera  and  the  light  source  as  one  unit  on  surfaces  1/3,  which 
required  a  relatively  large  surface  with  an  unchanging  Bidirectional  Reflectance 
Distribution  Function  (BRDF). 


Figure  1.  Indirect  photography  with  camera  field  of  view  dictated  by  point  of  illumination. 
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1.3  Research  Focus 


In  an  effort  to  increase  the  practical  application  of  indirect  photography,  this 
research  focuses  on  the  possibility  of  moving  the  light  source,  while  keeping  the  camera 
FOV  immobile.  This  results  in  a  configuration,  shown  in  Figure  2,  where  the  laser  spots 
are  located  around  the  camera  field  of  view,  thus  reducing  both  the  area  needed,  as  well 
as  the  chances  that  the  reflective  properties  differ  at  various  locations  of  surface  1/3. 

In  this  case,  the  large  center  grid  again  represents  the  pixilated  field  of  view  of  the 
camera.  Using  this  setup,  this  is  constant  for  all  trials  and  the  laser  spots,  represented  by 
the  outer  ring  of  smaller  squares,  are  the  only  thing  which  changes  between  different 
trials. 


Figure  2.  Indirect  photography  setup  with  fixed  camera  field  of  view. 


This  research  also  focuses  on  the  matrix  formulations  of  light  transport  in  an 
effort  to  discover  the  general  form  of  geometrical  relationships  between  surfaces  1 ,  2  and 
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3,  ideally  leading  to  future  work  which  will  then  be  able  to  generalize  appropriate 
approximations  of  these  matrices  for  unknown  geometries. 

1.4  Investigative  Questions 

This  research  aims  to  answer  several  questions.  The  highest  priority  is  to 
determine  if  it  is  possible  to  resolve  an  image  of  an  object  indirectly  by  using  matrices  as 
a  means  of  representing  the  radiometric  equations  which  have  previously  been  developed 
(Hoelscher,  2011).  This  is  more  difficult  than  might  be  anticipated  due  to  the  nature  of 
the  matrices  involved  in  this  process.  Because  the  dimensions  of  the  matrices  correspond 
to  the  number  of  laser  spots  used,  the  desired  resolution  of  the  object  and  the  number  of 
pixels  in  the  camera,  in  most  cases  these  matrices  will  not  be  square,  making  them  non- 
invertible  by  simple  means. 

As  a  result,  it  is  also  important  to  determine  which  method  of  inverting  non¬ 
square  matrices  should  be  used.  Some  possibilities  include  using  the  Moore-Penrose 
pseudoinverse  or  matrix  right  divide  as  defined  in  Matlab.  If  neither  of  these  is  effective, 
it  will  also  necessitate  more  research  to  determine  if  other  code  will  need  to  be  written  to 
perform  the  desired  function. 

Even  after  the  preferred  mathematical  method  is  determined,  many  variables  still 
remain  which  can  affect  how  accurately  an  image  can  be  resolved.  These  include  the 
number  of  laser  spots  used  to  illuminate  surface  1,  as  well  as  the  general  location  and 
layout  of  these  laser  spots.  Since  the  ideal  number,  location,  and  layout  of  laser  spots 
will  likely  vary  as  a  result  of  changing  distances,  angles  and  desired  resolution,  the  first 
question  in  this  area  which  needs  to  be  answered  is  if  there  is  a  general  trend  in  these 
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matters.  Another  important  question  is  whether  there  is  some  sort  of  indicator  which 
could  be  used  to  quickly  find  the  ideal  setup  of  laser  spots,  rather  than  having  to 
repeatedly  simulate  any  given  situation  to  find  the  best  setup. 

All  of  this  research  is  initially  accomplished  with  a  known  geometry  in  order  to 
best  compare  the  resulting  image  with  the  original.  However,  as  knowing  the  exact 
location  and  geometry  of  a  hidden  object  is  not  realistic  in  most  cases,  another  point  of 
emphasis  will  be  to  understand  the  basic  form  which  various  matrices  take  in  order  to 
apply  these  to  unknown  geometries.  This  process  was  also  intended  to  determine  the 
importance  of  knowing  an  accurate  geometry  and  the  degradation  of  the  image  which 
occurs  when  the  geometric  relationship  between  surfaces  is  approximated,  rather  than 
known.  This  would  also  include  an  understanding  of  which  values  were  important  to 
know  exactly  and  which  can  be  estimated,  as  well  as  the  effect  when  the  geometry  is 
incorrect  or  the  data  from  the  camera  is  flawed. 

1.5  Implications 

Research  into  indirect  photography  is  still  in  its  relative  infancy.  As  a  result,  most 
practical  applications  will  not  be  seen  for  quite  some  time.  However,  once  enough 
research  is  completed,  indirect  photography  will  have  major  implications  in  a  variety  of 
fields,  particularly  to  the  military  professional.  With  the  current  state  of  mobile 
computing,  it  is  unlikely  that  this  technique  could  be  used  regularly  for  immediate  results. 
The  delay  involved  in  computing  makes  this  more  suitable  to  larger  scale  reconnaissance 
without  immediate  consequences  (such  as  identifying  if  a  particular  vehicle  is  present  in  a 
covered  hanger  or  not).  Later,  as  computing  power  increases,  indirect  photography  could 
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also  be  used  on  a  personal  level,  enabling  soldiers  to  see  what  awaits  them  around  a 
comer.  The  techniques  researched  here  could  also  be  applied  in  similar  situations  by 
police  officers,  as  well  as  by  firefighters  or  other  rescue  workers  in  order  to  determine  the 
locations  of  unconscious  or  otherwise  incapacitated  individuals. 

1.6  Document  Overview 

This  document  contains  a  short  summary  of  previous  work  in  the  field  of  indirect 
photography,  as  well  as  the  principles  upon  which  this  work  is  based.  Also  included  is  a 
discussion  of  the  research  performed,  both  through  experiment  and  simulation,  including 
the  specific  geometries  used  for  these.  The  data  gathered  from  the  experiment  and 
multiple  simulations  is  then  analyzed  to  search  for  trends  which  may  be  used  to 
determine  the  feasibility  of  using  a  matrix-based  technique  to  determine  the  reflectance  of 
a  hidden  object.  This  includes  determining  the  effects  of  changing  the  number  and 
position  of  these  laser  spots  and  surface  1,  and  the  method  of  matrix  inversion,  as  well  as 
a  brief  discussion  of  the  effect  of  incorrect  measurements.  This  information  is  then  used 
to  draw  conclusions  about  the  feasibility  of  matrix-based  determination  techniques  in 
indirect  photography. 
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2.  Literature  Review 


2.1  Chapter  Overview 

The  purpose  of  this  chapter  is  to  give  a  brief  insight  into  past  research  in  the  field 
of  indirect  photography  and  other  relevant  topics.  Because  indirect  photography  is 
relatively  new,  there  has  not  yet  been  a  great  deal  of  research  on  this  topic.  While  a  brief 
overview  of  related  research  is  included  here,  the  dissertation  “ Restoration  of  scene 
information  reflected  from  non-specular  media ”  by  Lieutenant  Colonel  Mark  Hoelscher 
(Hoelscher,  201 1)  is  recommended  for  a  more  in-depth  treatment  of  the  historical 
background  leading  up  to  indirect  imaging. 

Indirect  photography  is  an  adaptation  of  dual  photography  which  relies  on 
principles  of  Helmholtz  reciprocity.  As  a  result,  the  prior  research  considered  here  deals 
first  very  briefly  with  Helmholtz  reciprocity,  then  transitions  to  dual  photography,  before 
touching  briefly  on  the  prior  work  in  indirect  photography  accomplished  by  LtCol 
Hoelscher.  In  addition,  because  of  the  matrix  nature  of  this  research,  a  large  amount  of 
linear  algebra  is  required,  particularly  relating  to  matrix  inversion.  For  this  reason,  a 
brief  discussion  of  matrix  inversion  will  be  included  at  the  conclusion  of  this  chapter. 

2.2  Relevant  Research 

Helmholtz  reciprocity  states  that  light  (or  any  electromagnetic  radiation)  can  be 
reversed  without  changing  the  physics  involved  in  its  transportation.  Initially  Helmholtz 
only  determined  this  to  be  the  case  for  specular  (mirrored)  surfaces  (von  Helmholtz  & 
Southall,  1862).  However,  Raleigh  later  expanded  this  to  include  non-specular  media. 
(Rayleigh,  1900).  He  showed  that  given  a  single  surface  with  an  energy  source  situated 
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at  any  distance  and  at  any  angle  from  this  surface,  and  a  sensor  also  set  at  any  distance 
and  any  angle  (not  necessarily  the  same  distance  and/or  angle)  from  this  surface,  the 
source  and  the  sensor  can  be  interchanged  and  the  sensor  will  still  record  the  same 
measurement.  This  remained  the  case  even  if  the  light  underwent  reflection,  refraction, 
or  any  other  changes  in  path,  so  long  as  the  polarization  remained  unchanged.  This 
ability  to  mathematically  change  the  locations  of  source  and  sensor  led  to  the 
development  of  a  technique  referred  to  as  dual  photography  (Sen  et  al.,  2005). 

2.2.1  Dual  Photography 

Dual  photography  was  initially  pioneered  in  2005  by  Sen  et  al.  with  the  intent  of 
aiding  in  the  creation  of  computer  generated  graphics.  Though  Helmholtz  reciprocity 
was  well  understood  previous  to  this  time,  these  authors  were  able  to  apply  this  principle 
to  allow  images  to  be  viewed  from  the  locations  of  pixilated  light  sources.  For  a  full 
understanding  of  dual  photography,  the  reader  is  recommended  Dual  Photography  by 
Sen  et  al.  (Sen  et  al.,  2005). 

The  principles  used  in  developing  a  transport  matrix  will  receive  a  fuller  treatment 
here,  however,  as  these  were  applied  in  the  current  research  as  well.  Given  a  projector 
with  pxq  pixel  elements  shining  onto  some  surface  and  being  photographed  or  otherwise 
viewed  by  a  camera  or  sensor  with  mxn  pixel  elements,  each  element  of  the  projector  is 
lit  up  individually.  This  yields  pxq  trials,  each  of  which  are  captured  by  every  one  of  the 
camera’s  mxn  pixel  elements.  The  information  collected  by  the  camera  from  every  trial 
is  then  combined  as  shown  in  Figure  3  to  yield  a  matrix  with  pxq  columns  and  mxn  rows 
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representing  a  mapping  of  the  flux  from  each  of  the  pxq  projector  pixels  to  each  of  the 
mxn  camera  pixels. 


Figure  3.  Creating  the  transport  matrix  from  data  images  (Hoelscher,  2011). 


The  projected  pattern  is  defined  as  a  row  vector  with  pxq  columns  and  the  camera 
images  are  defined  as  a  column  vector  with  mxn  rows.  Thus,  the  matrix  representation  is 

d  =  Tp'  (2.1) 

where  c  ’  is  the  row  vector  of  the  camera  images,  p  ’  is  the  row  vector  of  the  projected 
pattern  and  T  is  the  transport  matrix.  The  authors  in  the  original  paper  formed  a  dual 
image  simply  by  transposing  T  such  that  p  ’  ’  would  be  a  virtual  image  formed  at  the 
projector  given  the  camera  projected  a  pattern  c  ’  Note,  however,  that  TTt  =£  1.  In  this 
thesis,  the  inverse  of  T,  T-1  will  be  required. 
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2.2.2  Indirect  Photography 

Indirect  photography  was  introduced  by  LtCol  Mark  Hoelscher  in  201 1  in  an 
effort  to  be  able  to  gain  access  to  information  which  is  accessible  by  neither  the  camera 
nor  the  light  source  (Hoelscher,  2011).  Here  the  idea  of  using  a  laser  rather  than  a 
pixilated  projector  as  the  source  of  illumination  was  first  introduced.  These  efforts  met 
with  considerable  success,  as  evidenced  in  Figure  4. 
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Figure  4.  5  of  clubs  indirect  images  at  various  resolutions  (a)  95x63,  (b)  47x31,  (c)  23x15, 

(d)  11x7,  (e)  5x3  and  (f)  5x3  dual  image  (Hoelscher,  2011). 
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This  research  made  the  decision  to  collocate  the  laser  and  the  camera  so  that  the 


laser  spot  and  the  camera  FOV  would  move  together  on  surfaces  1/3.  As  it  was  no  longer 
possible  to  simply  determine  T  by  observation,  this  would  need  to  be  modeled  using 
radiometric  principles.  A  large  amount  of  the  improvement  in  this  process  was  due  to  the 
use  of  a  blind  deconvolution,  which  was  only  possible  due  to  the  fixed  relationship 
between  laser  spot  and  camera  FOV  and  the  resulting  form  of  the  radiometric  equations 
used.  (Hoelscher,  2011). 

In  addition,  this  paper  suggested  using  a  matrix  formulation  as  a  means  of 
revealing  symmetries  which  might  exist  and  aid  in  the  deconvolution  process.  While 
this  idea  is  not  pursued  in  depth  in  his  work,  a  matrix  equation  is  suggested  in  equation 
(98)  of  the  dissertation  “ Restoration  of  scene  information  reflected  from  non-specular 
media ”  (Hoelscher,  2011).  Since  a  substantially  different  matrix  formulation  is  applied 
here,  the  form  suggested  by  LtCol  Hoelscher  is  not  included  here,  and  those  wishing  to 
compare  the  two  are  suggested  to  review  his  work. 

Additional  work  with  an  emphasis  on  a  fixed  camera  FOV  on  surfaces  1/3  was 
also  completed  using  data  collected  by  Nathan  Powell  and  analyzed  by  Lieutenants 
Simon  Ferrel  and  Jessica  Shafer  (Ferrel,  Schafer,  &  Powell,  201 1).  The  experimental 
setup  included  a  helium  neon  laser,  two  adjustable  mirrors,  two  non-specular  reflecting 
surfaces,  and  a  digital  camera,  arranged  as  shown  in  Figure  5.  The  laser  was  designed  to 
be  incident  upon  various  locations  on  the  surface  marked  as  surface  1,  and  was  controlled 
by  changing  the  angle  of  the  mirrors  in  the  system.  The  hidden  object  was  placed  face¬ 
down  on  surface  2,  and  the  camera  was  situated  to  take  a  photograph  of  surface  3. 


11 


Though  surface  1  and  surface  3  were  collocated  in  this  case,  the  location  of  the  incident 
laser  spots  was  kept  out  of  the  field  of  view  by  approximately  2”. 


Surface  2 
(Hidden  Object) 


Surface  1/ 
Surface  3 


Mirror  2 

_  Digital 
Camera 


Mirror  1 


HeNe  Laser 


Figure  5.  Prior  experimental  setup  (Ferrel,  Schafer,  &  Powell,  2011). 


The  laser  light  was  reflected  first  off  the  two  mirrors  before  hitting  the  first 
surface,  then  was  reflected  off  of  the  second  surface  containing  an  object  hidden  from  the 
camera.  In  an  effort  to  determine  the  resolving  power  of  indirect  photography  techniques 
for  this  setup,  the  hidden  objects  which  were  used  were  four  black  and  white 
checkerboards  of  increasing  spatial  frequency  as  shown  in  Figure  6. 


Figure  6.  Checkerboards  used  as  hidden  object. 
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From  this  hidden  object,  the  light  was  then  reflected  once  more  off  of  surface  3. 
The  camera  finally  collected  the  light  reflected  from  surface  3  which  was  within  its  field 
of  view.  An  example  of  such  a  photograph  is  shown  in  Figure  7. 


Figure  7.  Example  of  camera  image  for  experimental  setup. 

This  process  was  repeated  a  total  of  252  times,  varying  the  position  of  the  laser 
spot  slightly.  61  laser  spots  were  located  along  the  bottom  of  the  FOV,  with  an  additional 
63  on  each  side,  and  65  on  the  top,  all  spaced  approximately  equally  along  these  spaces. 
Each  laser  spot  presented  a  slightly  different  illumination  of  the  object,  and  thus,  a 
slightly  different  image  to  the  camera.  Combining  the  images  from  all  these  trials  as 
previously  described  in  Figure  3  yielded  the  intensity  matrix  used  in  this  case. 

Unfortunately,  no  recognizable  images  could  be  recovered  from  the  experimental 
data  regardless  of  the  spatial  frequency  of  the  hidden  object  or  the  method  used  for 
matrix  inversion.  One  representative  example  of  this  is  shown  in  Figure  8  for  an  attempt 
to  resolve  a  2x2  checkerboard  at  8x8  resolution  using  the  Moore-Penrose  pseudoinverse. 
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Figure  8.  Difference  between  ideal  image  (left)  and  recovered  image  (right)  in  experimental 
results  using  the  Moore-Penrose  pseudoinverse. 

While  the  matrix  inversion  resulted  in  some  data  being  irretrievably  lost,  the  main 
difficulties  were  a  result  of  backscatter  from  the  incident  laser  light.  This  was  a  result  of 
the  very  low  intensities  of  the  multiply  reflected  light  which  actually  contains  the  object 
information.  While  the  light  spots  were  outside  the  camera  FOY,  some  light  was 
reflected  back  off  surface  1  into  the  internal  workings  of  the  camera.  While  this  light  was 
not  directly  imaged  onto  the  CCD  in  the  camera,  scatter  off  the  lens  and  internal 
workings  of  the  camera  was  of  equal  or  greater  magnitude  than  the  light  containing  object 
information  and  was  processed  into  the  images. 

2.2.3  Linear  Algebra 

Given  an  mxn  matrix  A,  with  m  =£  n,  there  is  no  generalized  inverse  A~x  such  that 
AA~X  =  Im  and  A~x A  =  In,  where  Im  and  Im  are  the  identity  matrices.  For  the  cases  that 
the  square  matrices  AT A  and  AAT  are  invertible  there  does  exist  a  left  inverse  AJ^t  = 

C AtA)~1At  for  matrices  with  m>  n  and  a  right  inverse  A~lght  =  AT(AAT)_1  for 
matrices  with  m  <  n.  Because  these  are  limited  to  cases  where  the  matrix  is  not  rank 
deficient,  the  Moore-Penrose  pseudoinverse,  A*,  was  developed,  which  always  exists, 
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coincides  with  left,  right  and  general  inverses  when  they  exist,  and  has  the  following 
properties  (Penrose,  1955): 

•  AA^A  =  A  •  (AAf)*  =  AA + 

•  A+AA+  =  4+  •  (AUy  =  A*  A  (2-2) 

This  is  particularly  useful  for  cases  where  the  same  algorithm  needs  to  be  applied 
regardless  of  the  dimensions  of  a  given  matrix.  Another  method  of  matrix  inversion 
involves  using  the  matrix  right  divide,  mrdivide,  command  in  Matlab,  which  solves  the 
matrix  equation  A  =  XB  for  X  with  as  few  non-zero  elements  as  possible  (Leon,  2005). 

2.3  Summary 

There  has  been  very  limited  research  in  the  field  of  indirect  photography,  and  this 
research  has  occurred  only  recently.  Additionally,  while  the  work  performed  here  does 
build  upon  previous  advancements  in  understanding  Helmholtz  reciprocity,  dual 
photography,  indirect  photography,  and  linear  algebra,  it  nonetheless  differs  substantially 
from  past  efforts  in  these  fields. 
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3.  Methodology 


3.1  Chapter  Overview 

This  experiment  was  conducted  in  simulation  using  Matlab.  This  research  was 
modeled  after  the  experimental  trial  using  fixed  camera  FOV  on  surface  1  as  discussed  in 
Chapter  2.  (Ferrel,  Schafer,  &  Powell,  201 1).  This  involved  creating  a  simulation  with 
surfaces  1/3  and  2  in  a  specific  configuration,  and  testing  the  effects  on  the  returned 
indirect  image  which  resulted  from  varying  the  number,  location  and  layout  of  laser  spots 
on  surface  1 . 


3.2  Theory 

The  radiometric  equation  for  radiance,  from  which  all  other  radiometric  quantities 
can  be  derived,  is  given  by  the  spectral  flux  radiated  per  projected  unit  area  of  the  source 
per  detector  solid  angle,  or 


4 


dAsCOsOsdHd 


(W  cm  2sr  -1) 


(3.1) 


where  d<pe  is  the  differential  flux,  dAscos9s  is  the  differential  projected  area  of  the 
source,  and  dfld  the  differential  solid  angle  subtended  by  the  detector.  Rearranging  terms 
in  this  equation  and  performing  various  operations  give  the  other  radiometric  quantities, 
including  flux,  intensity,  exitance,  and  irradiance  (Dereniak  &  Boreman,  1996). 

Given  this,  if  the  initial  irradiance  is  known,  so  long  as  the  geometry  terms  and 
BRDFs  are  understood,  the  reflected  radiance  and  subsequent  irradiance  onto  other 
surfaces  may  both  be  found  after  arbitrarily  many  reflections.  Indirect  photography 
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involves  three  occasions  in  which  surfaces  are  impacted  by  light.  The  setup  studied  here 
is  shown  in  Figure  9. 


Figure  9.  The  spatial  coordinate  system  with  respect  to  the  three  reflecting  surfaces. 

The  coordinates  ( 'xj,yj )  correspond  to  the  positions  of  the  initial  laser  spots  on  surface  1. 
Coordinates  (x2.y2.z2)  refer  to  a  grid  of  locations  on  the  hidden  object  found  on  surface  2, 
while  (x,3,y3)  describe  the  grid  of  locations  on  the  surface  3  (which  in  this  case  is  the  same 
as  surface  1)  within  the  camera’s  field  of  view.  In  all  cases,  the  Cartesian  origin  was 
chosen  to  be  the  lower  left  comer  of  surface  1 . 

The  proportion  of  flux  which  reaches  surface  2  from  surface  1  depends  on  the 
BRDF  of  surface  1  with  respect  to  surface  2  (Driggers,  Cox,  &  Edwards,  1999)  and  is 
proportional  to 
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(3.2) 


\cosddl2  cosdsl2 


R 


12 


|  O  [ BRDF12(9i1,9sl2,A012 )] 


where  9d 12is  the  zenith  angle  between  a  point  on  surface  1  (SI)  and  a  point  on  surface  2 
(S2)  with  respect  to  the  normal  of  S2.  0sl2is  the  zenith  angle  between  a  point  on  surface 
1  (SI)  and  a  point  on  surface  2  (S2)  with  respect  to  the  normal  of  SI.  R12  is  the  distance 
between  these  two  points.  [BRDF12  (0;i,  0sl2,  A012)]  is  the  matrix  of  BRDFs  of  points 
on  SI  to  points  on  S2,  which  depends  on  the  incident  angle  9^,  the  zenith  angle  between 
the  source  and  a  point  on  SI  with  respect  to  the  normal  of  SI,  0sl2and  A012,  the 
azimuthal  angle  between  the  projection  of  the  source  ray  onto  SI  and  the  projection  onto 
SI  of  the  ray  between  points  on  SI  and  S2.  O  is  the  Hadamard  product,  commonly 
known  as  point-wise  matrix  multiplication  (Hogban,  Brualdi,  Greenbaum,  & 
Matthias,  2003). 

Similarly,  the  proportion  of  flux  which  reaches  surface  3  and  then  the  camera 
from  surface  2  is  given  by 


cos  9d 23  cos  9 


S23 


R 


23 


O  [B  RDF23cam(9  d23,  @s3cam>  ^03  cam)] 


O  [BRDF123(9dl2,  9s23,  A023^] 


(3.3) 


with  similar  notation  as  Equation  (3.2).  Here,  the  subscripts  on  BRDF  indicate  position 
on  the  surface  that  is  the  source  of  the  irradiance,  the  reflecting  surface  being  considered, 
and  position  on  surface  collecting  the  reflected  radiance,  respectively.  It  should  be  noted 
that  in  both  these  equations,  cos  9d23,  cos  9s23  and  R23  are  dependent  upon  the  exact 
location  of  the  resolution  element  on  surface  2  and  camera  FOV  pixel  on  the  surface  3, 
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and  thus  take  different  values  for  every  camera  pixel  and  every  resolution  element  of  the 
hidden  object.  Similarly,  cos  ddl2,  cos  0sl2  and  R12  take  unique  values  for  every  laser 
spot  position  and  every  resolution  element  of  the  hidden  object. 

This  allows  this  total  series  of  laser  spot  movement  for  the  system  of  experiments 
shown  in  Figure  9  to  be  represented  by  the  following  matrix  system: 


[^caml  —  ^laser^s2^lens^FOV 


[BRDF23cam]  O  [BRDF12 3]  O  [cosOd 23]  O  [cos  0S 23\  Q 
1 


*23 


(3.4) 


*12. 


o  [BRDF12]  o  [cos0dl2]  o  [ cosdsl2 ] 


where  [(t,cam]  is  a  matrix  the  flux  measured  by  the  camera  for  any  pixel  at  S3  for  any 
laser  spot  position  on  SI,  0/aser  is  the  laser  flux,  i4s2  is  the  resolution  area  defined  on  S2, 
A  iens  is  the  area  of  the  camera  lens,  flF0V  is  the  solid  angle  FOV  of  each  camera  pixel, 
and  *  denotes  the  Kroenecker  product,  which  is  a  general  matrix  multiplication  (Hogban, 
Brualdi,  Greenbaum,  &  Matthias,  2003).  0/aser,  As2,  Alens,  and  F1fov  are  considered 
constants  in  these  experiments.  Since  the  absolute  magnitude  of  [Ocam]  is  unimportant, 
these  four  can  be  considered  to  just  scale  [$cam]  and  therefore  removed  from  the 
equation. 

In  order  to  further  simplify  Equation  (3.4),  the  assumption  is  made  that  the 
surfaces  1  and  3,  upon  which  the  laser  light  will  be  incident,  are  approximately 
Lambertian  and  therefore  perfectly  diffuse.  While  this  is  not  exactly  the  case,  this 
approximation  will  allow  BRDF23cam  and  BRDF12  to  be  approximated  as  constants. 
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Additionally  BRDF123 is  a  three  dimensional  matrix  because  it  accounts  for  the  laser  spot 
positions  on  SI,  the  resolution  elements  on  S2,  and  the  individual  pixel  locations  of  the 
camera’s  FOV  on  S3.  Since  solution  of  Equation  (3.4)  only  allows  for  a  two  dimensional 
matrix  BRDF123,  the  assumption  must  be  made  that  the  BRDF  at  any  position  on  SI  for 
any  reflectance  angle  to  S3  is  the  same  regardless  of  the  incident  angle  from  SI.  This  is  a 
good  assumption  for  a  Lambertian  surface.  Because  of  this,  the  only  non-constant  terms 
remaining  in  this  matrix  equation  are  those  geometry  terms  which  depend  on  the 
separations  and  angles  between  various  points,  as  well  as  BRDF123,  which  represents  the 
object.  Therefore,  Equation  (3.4)  simplifies  to 

[O 

cam  ]  oc  {[BRDF123\  o  [G23]}  *  [G12]  (3.5) 

where  [G23]  represents  the  geometric  terms  between  the  resolution  points  on  surface  2 
and  the  pixels  of  the  camera  FOV  on  surface  3,  and  is  given  by 

[623]  =  [cOS0d23]  O  [cos0s23]  O  (3-6) 

with  all  matrices  of  dimension  pxr  where  p  is  the  number  of  camera  pixels  and  r  the 
number  of  resolution  elements.  Similarly,  [ G12\  represents  the  geometric  relationship 
between  the  laser  spots  on  surface  1  and  the  resolution  points  on  surface  2,  and  is  given 
by 

[G12]  =  [cosddl2]  ©  [cos  0sl2]  ©  [^-].  (3.7) 
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with  all  matrices  of  dimension  rxl  where  r  is  again  the  number  of  resolution  elements  and 
/  corresponds  to  the  number  of  laser  spots  used.  These  dimensions  cause  the  recorded 
flux  matrix  [<bcam]  to  have  dimensions  pxl,  which  matches  the  method  shown  in  Figure  3 
for  creating  this  matrix.  While  some  of  these  geometric  terms  are  generally  unknown,  in 
this  particular  experiment  it  is  possible  to  measure  them  as  a  result  of  knowing  locations 
of  all  points  of  interest.  As  a  result,  the  BRDF  of  the  unseen  object  can  theoretically  be 
determined  using  matrix  manipulation.  This  requires  first  that  an  inverse  be  found  for 
[G12],  so  that  [<t>cam]  can  be  multiplied  by  it  to  give 

[O 

cam  ]  *  [^12]  1  K  [5i?DF123]  O  [G23]  (3-8) 

Following  this,  [C23]  is  easily  accounted  for  by  simply  taking  the  entry-wise  reciprocal 
and  multiplying  both  sides  by  that  matrix.  Since  this  step  does  not  introduce  additional 
uncertainty,  the  majority  of  the  mathematical  research  done  in  this  case  centers  on  the 
inversion  of  [C12]. 

3.3  Simulated  Setup 

The  simulation  was  modeled  after  the  setup  used  by  Ferrel,  Schafer  and  Powell 

described  in  Chapter  2  and  with  schamtic  shown  in  Figure  9  (Ferrel,  Schafer,  &  Powell, 

2011).  In  this  case,  the  angle  between  surface  1/3  and  surface  2  is  45  degrees,  with 

surface  1/3  having  a  total  length  of  15.5”  and  a  width  of  1 1 .5”.  The  hidden  object  was 

placed  on  surface  2,  with  the  rear-most  portion  of  the  image  5.87”  from  the  position 

where  surface  2  intersects  surface  1.  The  camera  FOV  remained  in  the  same  position 

regardless  of  location  of  the  laser  spot.  Though  the  camera  itself  had  a  rectangular  FOV, 
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due  to  the  angle  of  surface  1/3,  the  FOV  measured  4”  in  both  the  x-direction  (up  the 
board)  and  the  y-direction  (across  the  board),  with  the  lower-left  comer  of  the  FOV 
located  at  coordinates  corresponding  to  3.25”  in  the  x-direction  and  2.75”  in  the  y- 
direction. 

In  an  effort  to  determine  the  resolving  power  of  indirect  photography  techniques 
for  this  setup,  the  hidden  objects  which  were  used  were  the  same  four  black  and  white 
checkerboards  of  increasing  spatial  frequency  shown  in  Figure  6.  Each  of  these 
checkerboards  was  of  size  2”x2”,  with  the  rear-most  portion  of  the  checkerboard  5.87” 
directly  above  SI  and  the  front  of  the  checkerboard  9.87”  above  SI.  The  Matlab  code 
used  to  model  these  positions,  as  well  as  the  position  of  the  laser  spots,  can  be  found  in 
Appendix  A. 

Because  these  trials  were  accomplished  in  simulation,  rather  than  being  able  to 
simply  photograph  the  resulting  intensity  on  surface  3,  the  matrix  equation  found  in 
Equation  (3.5)  was  used  to  determine  the  intensity  that  would  be  photographed  by  a 
camera  in  the  ideal  case.  This  implies  that  there  are  no  aberrations  or  any  other  factors 
which  disturb  the  transport  of  light  except  for  the  non-specular  reflection  at  the  three 
surfaces.  While  this  is  not  entirely  realistic,  it  is  useful  in  evaluating  the  fundamental 
capabilities  and  limitations  of  matrix-based  formulations  of  indirect  photography. 

The  quality  of  the  resulting  indirect  images  is  quantified  by  the  modified 
Modulation  Transfer  Function  (MTF)  introduced  by  LtCol  Hoelscher  as: 


MTF  = 


White-Black 

White+Black 


(3.9) 


22 


where  White  is  the  average  reflectance  of  all  pixels  in  a  recovered  indirect  image  which 
correspond  to  the  white  pixels  of  the  object  and  Black  is  the  average  reflectance  of  all 
pixels  in  a  recovered  indirect  image  which  correspond  to  the  black  pixels  of  the  object 
(Hoelscher,  2011).  An  MTF  of -1 .0  would  be  the  exact  opposite  image  of  the  object,  an 
MTF  of  0.0  would  show  no  resolution  of  the  objects  spatial  features  and  an  MTF  of  1.0 
would  be  a  perfect  image  of  the  object.  MTF  values  less  than  -1 .0  or  greater  than  1 .0  can 
only  occur  mathematically  in  the  case  that  some  pixels  take  negative  reflectance  values, 
which  is  not  physical. 

This  evaluation  via  MTF  occurred  for  four  different  variables  modified  in  Matlab 
code.  These  were  (1)  changing  how  the  pseudoinverse  of  a  matrix  is  calculated,  (2) 
changing  the  number  of  laser  spots  on  surface  1,  (3)  changing  the  distance  that  the  laser 
spots  are  from  the  camera  field  of  view  on  surface  1/3  and  (4)  changing  how  the  laser 
spots  are  arranged  at  that  distance.  An  example  of  two  different  laser  spot  layouts  is 
shown  in  Figure  10. 
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Figure  10.  Different  laser  spot  layouts. 
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Additionally  some  analysis  was  made  of  the  effect  of  small  errors  in  the  data 
recorded  by  the  camera,  by  purposely  introducing  a  random  normal  percent  error  with 
varying  standard  deviation  to  the  correctly  derived  intensity.  All  of  these  variables  were 
not  changed  simultaneously  in  an  effort  to  determine  an  overall  best-case  scenario  as  the 
currently  available  computing  power  made  it  infeasible  to  consider  every  possible 
permutation  of  these.  Instead,  each  is  considered  individually  in  the  hopes  of  unearthing 
general  trends. 

While  a  variety  of  different  Matlab  code  was  used  in  changing  these  different 
variables,  one  representative  example  is  included  in  Appendix  A. 

3.4  Chapter  Summary 

The  experiment  described  in  Chapter  2  and  the  simulated  portion  of  this  research 
both  use  the  same  geometry,  setup  and  equations,  have  different  purposes.  The 
experiment  was  designed  to  both  test  the  real-world  application  of  matrix-based 
techniques  of  indirect  photography  and  expose  any  fundamental  issues  with  the  design  of 
the  experiment.  Its  immediate  limitation,  described  in  Chapter  2,  prompted  this  research 
to  be  conducted  via  simulation,  which  allows  for  more  trials  as  well  as  serves  to  remove 
extraneous  variables  which  might  make  it  more  difficult  to  gain  an  understanding  of  the 
fundamental  concepts  involved  in  using  matrices  to  perform  indirect  photography. 
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4.  Analysis  and  Results 


4.1  Chapter  Overview 

In  simulation,  a  matrix-based  technique  for  indirect  imaging  was  shown  to  recover 
recognizable  images.  The  quality  of  these  images  varied  greatly  depending  on  the 
desired  resolution  and  the  various  controllable  aspects  of  the  setup,  such  as  method  of 
matrix  inversion,  and  number  of  laser  spots  and  the  location/layout  of  laser  spots  on 
surface  1 .  General  trends  were  found  as  to  which  setups  would  be  the  most  effective,  and 
multiple  efforts  were  made  to  find  an  indicator,  such  as  condition  number  or  eigenvalue 
of  the  \G12\  matrix,  which  would  show  which  setup  is  most  effective  without  requiring 
multiple  simulation  runs.  All  possible  attempts  at  this  showed  no  correlation  to  the 
effectiveness  in  resolving  the  image.  Nonetheless,  these  general  trends  give  some  insight 
as  to  methods  of  improving  images  acquired  via  indirect  photography. 

4.2  Results  of  Simulation  Scenarios 

A  number  of  variables,  all  of  which  are  under  user  control,  were  considered  in 
trying  to  maximize  the  modified  MTF,  as  defined  by  Equation  (3.9),  of  the  acquired 
image.  These  included  the  method  of  matrix  inversion,  as  well  as  the  number  and 
positions  of  the  laser  spots  on  surface  1 .  Additionally,  simulation  was  performed  to 
determine  the  effect  of  small  errors  in  flux  as  measured  by  the  camera.  Because  of 
computing  power,  these  variables  could  not  be  considered  jointly,  but  each  is  considered 
individually  and  yields  usable  information  and  a  general  trend. 
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4.2.1  Testing  Method  of  Matrix  Inversion 


There  are  two  different  ways  to  solve  the  matrix  equation  A  =  XB  when  B  is  a 
non-square  matrix.  This  does  not  include  using  the  right  inverse  B~ight  =  BT (BBT')~1  ■ 
Since  the  Moore-Penrose  pseudoinverse  is  equal  to  the  right  matrix  inverse  when  the 
right  inverse  exists,  testing  using  the  Moore-Penrose  pseudoinverse  makes  testing 
indirect  image  quality  using  the  right  matrix  inverse  unnecessary.  Using  the  Moore- 
Penrose  pseudoinverse  minimizes  the  Euclidean  norm  for  A  —  XB,  whereas  using  the 
mrdivide  Matlab  command  solves  with  as  sparse  a  matrix  as  possible.  The  more 
effective  method  is  not  initially  obvious,  especially  when  only  considering  the  modified 
MTF,  where  1.0  is  ideal.  Table  1  compares  the  modified  MTF  accomplished  by  each  of 
these  methods  for  resolution  of  a  2x2  checkerboard  with  64  resolution  elements  using  an 
increasing  number  of  laser  spots. 


Table  1.  Modified  MTF  of  2x2  checkerboard  with  64  resolution  elements  by  method 


#  of  laser  spots 

Moore-Penrose 

mrdivide 

80 

0.9714 

0.9208 

400 

0.9772 

0.9822 

4000 

0.9578 

0.9705 

By  this  comparison,  the  two  methods  seem  to  be  similarly  effective.  However,  in 
viewing  the  images  returned  by  each  of  these  methods,  shown  in  Figure  14,  it  is  obvious 
that  the  mrdivide  images  do  not  correspond  to  the  MTF  values  assigned  to  them. 
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Using  80  laser  spots 


Using  400  laser  spots 

Moore-Penrose  (MTF=0.9569)  ^^^nmiivide(MTF=0.9822) 


Using  4000  laser  spots 

Moore-Penrose  (MTF=0.9438)  _ mrdivide  (MTF=0.9705) 


I  I  I 


Figure  11.  Comparison  of  indirect  images  returned  for  2x2  checkerboard  objects  via 
Moore-Penrose  pseudoinverse  and  mrdivide. 
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This  disparity  between  image  quality  and  assigned  MTF  value  is  a  result  of 
mrdivide  returning  negative  reflectance  values,  which  are  non-physical  (Meyer-Spradow 
&  Loviscach,  2003).  The  Moore -Penrose  pseudoinverse  does  not  yield  negative  values 
as  solution  to  this  matrix  system  and  is  the  preferred  method  for  accurately  resolving  the 
image,  regardless  of  the  spatial  frequency  of  the  object  or  desired  resolution  of  the 
resulting  indirect  image.  For  this  reason,  all  remaining  tests  are  done  with  matrix 
inversion  accomplished  via  the  Moore-Penrose  pseudoinverse. 

4.2.2  Testing  Differing  Number  of  Laser  Spots 

One  of  the  other  variables  which  can  be  adjusted  from  the  position  of  the  operator 
at  the  collocated  camera  and  laser  is  the  number  of  laser  spots  used  for  illumination, 
which  corresponds  to  the  number  of  experiments  performed  in  order  to  obtain  an  indirect 
image.  Because  each  laser  spot  used  increases  the  number  of  trials  and  therefore  the 
dimensions  of  several  matrices  involved  in  the  calculation,  it  is  advisable  to  use  as  few 
laser  spots  as  possible  to  resolve  a  clear  image.  The  ideal  number  of  laser  spots  depends 
both  on  the  spatial  frequency  of  the  object,  as  well  as  the  desired  resolution,  as  can  be 
seen  in  Figures  12-14,  where  all  trials  were  performed  with  the  laser  spots  equally  spaced 
in  a  6”x6”  square  pattern  centered  on  the  point  on  S 1  directly  beneath  the  center  of  the 
hidden  object.  The  16x16  objects  are  not  included  since  they  cannot  be  recognizably 
resolved  regardless  of  laser  spot  setup.  In  viewing  these  images,  it  must  be  noted  that 
these  results  are  specific  to  the  geometry  used  in  these  simulated  trials  and  that  the  ideal 
number  of  laser  spots  will  likely  vary  with  the  setup. 
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(a)  (b) 


(c)  (d) 


Figure  12.  Modified  MTF  as  a  function  of  number  of  laser  spots  for  2x2  objects  at  different 
resolutions:  (a)  4x4,  (b)  8x8,  (c)  16x16,  (d)  24x24,  (e)  32x32  and  (f)  64x64. 
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Modified  MTF  Modified  MTF  Modified  MTF 


(a) 


(b) 


(c)  (d) 


(e)  (f) 


Figure  13.  Modified  MTF  as  a  function  of  number  of  laser  spots  for  4x4  objects  at  different 
resolutions:  (a)  4x4,  (b)  8x8,  (c)  16x16,  (d)  24x24,  (e)  32x32  and  (f)  64x64. 
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Figure  14.  Modified  MTF  as  a  function  of  number  of  laser  spots  for  8x8  objects  at  different 
resolutions. 


Generally,  however,  it  may  be  noted  that  modified  MTF  increases  with  the 
number  of  laser  spots  before  leveling  off,  and  that  a  higher  desired  resolution  will  require 
the  use  of  more  laser  spots  in  order  to  achieve  a  similar  quality  of  indirect  image.  It  may 
also  be  noted  that,  for  each  spatial  frequency  checkerboard,  the  value  of  the  modified 
MTF  tends  to  level  off,  regardless  of  the  desired  resolution.  This  may  be  indicative  of  the 
laser  spots  being  so  near  one  another  as  to  no  longer  give  substantially  different  angles  of 
illumination  on  the  object  and  information. 

Because  this  portion  of  the  experiment  was  conducted  entirely  in  simulation,  the 
only  cause  of  recovered  images  differing  from  the  actual  object  is  the  data  lost  due  to  the 
necessary  inversion  of  a  geometrical  matrix  to  solve  for  the  indirect  image  of  the  object. 
Since  this  matrix’s  dimensions  and  values  change  as  the  number  and  layout  of  the  laser 
spots  change,  ideally  some  characteristic  of  this  matrix  should  change  in  relation  to  the 
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MTF  of  the  recovered  images,  enabling  this  characteristic  to  serve  as  an  indicator  to 
select  the  best  setup  of  laser  spots. 

The  condition  number  of  a  matrix  is  defined  as  the  ratio  of  the  maximum  and 
minimum  eigenvalues  of  a  matrix  (Erceg,  Soma,  Baum,  &  Paulraj,  2002)  and  is  equal  to 
the  norm  of  a  matrix  multiplied  by  the  norm  of  the  matrix  inverse,  or  pseudoinverse  in 
this  case.  Though  condition  number  of  this  matrix,  which  is  independent  of  the  object, 
was  tested  as  a  possible  indicator  of  ideal  setup,  there  was  no  discernible  pattern  to  the 
condition  number,  as  shown  in  Figure  15.  Condition  number  also  serves  as  a  measure  of 
sensitivity  of  the  solution,  or  how  error  in  output  (in  this  case,  the  image  of  the  object) 
increases  as  a  function  of  error  in  input  (in  this  case,  the  data  measured  by  the  camera). 
Inversion  by  a  matrix  with  a  condition  number  of  1  does  not  increase  the  error,  whereas  a 
matrix  with  a  larger  condition  number  will  require  more  initial  accuracy.  A  large 
condition  number  also  indicates  that  the  matrix  is  less  invertible.  Given  the  very  large 
condition  numbers  seen  in  Figure  15,  particularly  at  higher  resolutions,  this  is  a  serious 
concern  and  is  addressed  in  section  4.3.5. 

Another  possible  indicator  would  be  to  consider  only  the  smallest  eigenvalue  of 
[C12]  [C12]\  which  is  a  square  matrix  regardless  of  the  dimension  of  [G12],  Matrices  are 
non-invertible  if  and  only  if  they  have  a  zero  eigenvalue  (Johnson,  1970).  Considering 
this,  a  near  zero  eigenvalue  might  indicate  that  the  matrix  is  near  singular.  As  can  be 
seen  in  Figure  16,  this  also  fails  to  correspond  to  changes  in  modified  MTF. 
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Figure  15.  Condition  number  as  a  function  of  number  of  laser  spots  for  (a)  2x2,  (b)  4x4, 
(c)  8x8  and  (d)  16x16  resolutions. 


Neither  the  condition  number  nor  the  minimum  eigenvalue  of  this  matrix 
correspond  to  the  changes  in  modified  MTF.  While  there  may  be  some  other  value 
derived  from  this  matrix  to  indicate  the  ideal  number  of  laser  spots,  it  has  not  been  found. 
As  a  result  determining  the  ideal  number  of  laser  spots  for  a  specific  geometry  requires 
an  exhaustive  simulation  prior  to  beginning  the  physical  setup. 
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Figure  16.  Minimum  eigenvalue  as  a  function  of  number  of  laser  spots  for  (a)  2x2,  (b)  4x4, 
(c)  8x8  and  (d)  16x16  resolution. 


4.2.3  Testing  Laser  Spots  at  Differing  Distances  from  Object 

Another  factor  which  can  be  varied  is  the  dimensions  of  the  square  pattern  of  laser 
spots  centered  at  a  point  on  SI  directly  beneath  the  hidden  object,  hereafter  denoted  as  C. 
The  ideal  location  is  not  with  the  sides  of  this  square  as  close  to  C  as  possible.  As  these 
laser  spots  move  further  from  C,  it  initially  causes  successive  laser  spot  location  to  result 
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in  a  greater  change  in  geometry  between  the  laser  spot  location  and  object  location, 
thereby  contributing  values  to  the  [ G12 ]  matrix  which  makes  it  less  singular.  Simulation 
was  used  to  determine  at  which  perpendicular  distances  from  this  center  point  C  the 
modified  MTF  was  maximized,  with  examples  shown  in  Figures  17  and  18. 


Figure  17.  Modified  MTF  as  a  function  of  laser  spot  distance  from  C  for  an  8x8  object  at 
8x8  resolution  with  100  laser  spots. 


One  possible  reason  for  the  general  trend  shown  in  Figure  17  again  involves  the 
invertibility  of  the  matrix  involved  in  these  calculations.  When  the  same  number  of  laser 
spots  is  put  into  a  smaller  perimeter  as  occurs  when  the  dimensions  of  a  rectangle  are 
decreased,  they  become  closer  to  one  another.  Since  each  column  of  this  matrix 
represents  the  geometric  relationship  of  a  single  laser  spot  on  surface  1  to  all  points  on 
surface  2,  the  matrix  can  become  nearly  singular  if  the  laser  spots  are  not  separated  by  an 
appropriate  distance. 
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1”  away- 0.1365  3”  away- 0.1715 


5”  away -0.4731  7.5”  away  -  0.5998 


9”  away- 0.3671  11”  away- 0.5333 


Figure  18.  Recovered  images  for  8x8  object  at  8x8  resolution  with  100  laser  spots  at  varying 
distances  from  C. 
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This  is  of  particular  interest  when  considering  the  horizontal  lines  of  laser  spots 
across  surface  1.  A  cross-sectional  view  of  the  experimental  setup  is  shown  in  Figure  19, 
which  allows  for  the  calculation  of  the  basic  relationship  between  these  lines  of  laser 
spots  and  the  various  x-direction  lines  of  the  hidden  object. 


r 


d 

j _ 

5.87" 


7.87" 


9^87" 


Figure  19.  Cross-section  of  setup  used  in  experimental  and  simulated  trials. 


The  law  of  sines  gives 


sin  (a)  _  sin  (/ 3 ) 
d  _  15.5— (Cy+Ay) 


(4.3.3. 1) 
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where  Cy  is  the  y-coordinate  of  the  point  C,  here  4.375”,  and  d  and  Ay  are  as  shown  in 
Figure  19.  Given  that  /?  =  135°  —  a,  a  can  be  defined  in  terms  of  Cy,  Ay  and  d  with 

a  =  arccot  (42  15  5  ^y+Ay^  —  1^.  (4.3. 3. 2 

The  angle  from  the  lower  horizontal  line  of  spots  is  given  by  the  same  expression  except 
with  Cy  +  Ay  replaced  by  Cy  —  Ay. 

From  Figure  17,  the  max  modified  MTF  with  a  rectangular  layout  for  an  8x8 
object  with  8x8  desired  resolution  occurs  when  the  laser  spots  are  7.5”  from  C.  By  using 
Equation  (4. 3. 3. 2)  to  solve  for  a,  it  is  possible  to  solve  for  simplified  versions  of  0sl2, 
ddl2 and  R12  for  the  flattened  cross-section,  denoted  as  0sl2, 0dl2  and  r12.  Letting 
K  =  cos  0sl2  cos  0dn/rl2,  the  geometric  term  in  this  flattened  cross-section,  allows 
for  comparison  of  this  quantity  between  different  laser  spot  positions,  as  seen  in 
Tables  2,  3  and  4. 


Table  2.  K  between  horizontal  lines  of  laserspots  and  rows  of  hidden  object  with  laser 
spots  in  square  pattern  with  edges  7.5”  from  C. _ _ 


Row  of  Hidden  Object 

(Row  1  is  rear-most  row,  Row  8  is  front-most) 

Ratio 

1 

2 

3 

4 

5 

6 

7 

8 

K  for  lower 
line  of  spots 

0.0011 

0.0013 

0.0015 

0.0017 

0.0019 

0.0021 

0.0024 

0.0026 

2.262 

K  for  upper 
line  of  spots 

0.0301 

0.0227 

0.0173 

0.0134 

0.0105 

0.0084 

0.0068 

0.0055 

5.474 

Ratio 

26.29 

17.21 

11.51 

7.87 

5.50 

3.93 

2.86 

2.12 
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Table  3.  K  between  horizontal  lines  of  laserspots  and  rows  of  hidden  object  with  laser 
spots  in  square  pattern  with  edges  1”  from  C. _ _ 


Row  of  Hidden  Object 

(Row  1  is  rear-most  row,  Row  8  is  front-most) 

Ratio 

1 

2 

3 

4 

5 

6 

7 

8 

K  for  lower 
line  of  spots 

0.0059 

0.0067 

0.0075 

0.0084 

0.0091 

0.0097 

0.0101 

0.0104 

1.787 

K  for  upper 
line  of  spots 

0.0113 

0.0126 

0.0137 

0.0146 

0.0151 

0.0153 

0.0152 

0.0148 

1.354 

Ratio 

1.93 

1.88 

1.82 

1.75 

1.67 

1.58 

1.50 

1.41 

Table  4.  K  between  horizontal  lines  of  laserspots  and  rows  of  hidden  object  with  laser 
spots  in  square  pattern  with  edges  11”  from  C. _ 


Row  of  Hidden  Object 

(Row  1  is  rear-most  row,  Row  8  is  front-most) 

Ratio 

1 

2 

3 

4 

5 

6 

7 

8 

K  for  lower 
line  of  spots 

0.0006 

0.0007 

0.0008 

0.0009 

0.0010 

0.0011 

0.0012 

0.0013 

2.240 

K  for  upper 
line  of  spots 

0.0003 

0.0002 

0.0002 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

3.968 

Ratio 

2.08 

3.02 

4.29 

5.95 

8.10 

10.83 

14.25 

18.48 

As  can  be  seen  from  the  ratios  in  the  far  right  column,  which  relate  the  largest  and 
smallest  values  of  K  for  a  given  line  of  laser  spots,  the  variation  in  flux  returned  from 
different  rows  of  the  object  is  greatest  when  the  line  of  laser  spots  is  located  7.5”  from  the 
center  point  C.  Additionally,  for  this  setup,  the  lower  and  upper  horizontal  lines  of  laser 
spots  also  shows  greater  differences  at  7.5”  than  the  other  distances  considered.  These 
traits  combine  to  give  the  matrix  more  variation,  which  makes  it  less  singular  and  more 
invertible,  resulting  in  improved  image  quality. 
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4.2.4  Testing  Differing  Layout  of  Laser  Spots 

None  of  the  previous  examples  were  able  to  achieve  a  modified  MTF  of  greater 
than  0.6  and  had  large  areas  in  the  middle  of  the  image  which  were  not  well  resolved. 
One  of  the  controllable  variables  which  makes  the  biggest  difference  in  eventual  image 
quality  is  the  layout  of  the  laser  spots.  As  an  example,  in  Figure  18,  the  image  acquired 
using  lines  of  laser  spots  5”  from  point  C  had  a  modified  MTF  value  of  0.4731.  Merely 
by  adding  a  slight  stagger  to  the  position  of  these  spots,  while  keeping  the  number  and 
general  position  the  same,  the  MTF  can  be  increased  to  1.00,  as  shown  in  Figures  20  and 
2 1 .  The  red  dots  in  Figure  20  (right)  correspond  to  the  experiment  without  stagger  and  to 
the  red  line  in  Figure  20  (left),  while  blue  corresponds  to  the  experiment  with  stagger. 


Figure  20.  (Left)  Modified  MTF  as  a  function  of  distance  from  C  for  two  different  patterns 
of  laser  spots;  (Right)  Laser  spot  patterns  used. 


Marked  improvements  such  as  this  are  accomplished  in  nearly  every  case  by 
staggering  the  laser  spots  as  shown  in  Figure  20.  The  top  of  Figure  21  shows  the  contour 
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maps  of  [C12],  the  matrix  requiring  inversion,  for  an  8x8  object  with  8x8  resolution,  with 
the  resulting  indirect  images  placed  underneath.  The  images  on  the  left  are  accomplished 
without  stagger  and  correspond  to  the  red  portions  of  Figure  20,  while  those  on  the  right 
make  use  of  staggered  laser  spot  location  and  correspond  to  the  blue  portions  of  Figure 
20.  Similar  to  the  image  improving  as  the  laser  spots  initially  move  away  from  the  field 
of  view  on  surface  1/3  it  is  possible  this  is  a  result  of  the  matrix  simply  becoming  less 
singular  due  to  increased  variation  in  the  position  of  the  laser  spots.  This  theory  is 
supported  by  the  top  images  in  Figure  21,  but  further  study  is  still  needed  on  this  matter. 


Figure  21.  (Top)  Contour  map  of  matrix  [G12]  and  (bottom)  resulting  images.  Left  is 
without  stagger,  right  is  with  stagger  of  laser  spot  layout. 
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4.2.5  Effect  of  Small  Error  in  Measurement 


By  adjusting  the  placement,  pattern  and  number  of  spots,  it  is  theoretically 
possible  to  get  a  perfect  copy  of  the  object  as  an  indirect  image,  as  shown  in  simulation. 
It  is  important,  however,  to  note  that  real-world  conditions  are  not  ideal,  and  there  will 
likely  be  some  systemic  error  in  those  cases.  The  extremely  large  condition  numbers  of 
matrices  to  be  inverted  make  this  a  particularly  important  case  to  consider.  Large 
condition  numbers  for  the  matrix  X  means  that  when  solving  AX  =  B  ford,  even  small 
errors  in  B  result  in  large  errors  in  A  (Zielke,  1983). 

To  determine  the  effect  that  detection  noise  or  other  systematic  uncertainties  can 
have  on  a  matrix  approach  to  indirect  photography,  Figure  22  shows  the  modified  MTF 
of  the  recovered  indirect  image  at  various  resolutions,  and  Figures  23  and  24  show  the 
recovered  image,  after  a  random  normally  distributed  percent  error  with  varying  standard 
deviations  has  been  individually  introduced  to  each  pixel  in  the  photographed  intensity. 


Figure  22.  Modified  MTF  of  2x2  objects  at  varying  resolution  (Blue  -  2x2,  Green  -  4x4,  Red 
8x8)  with  120  staggered  laser  spots  and  lines  6"  from  C. 
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Figure  23.  Recovered  indirect  images  for  2x2  object  at  4x4  resolution  with  intensity  added 
percent  error  with  standard  deviation  of  (a)  10,  (b)  1,  (c)  0.1,  (d)  le-2,  (e)  le-3  and 
(f)  No  error. 
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Figure  24.  Recovered  indirect  images  for  2x2  object  at  8x8  resolution  with  intensity  added 
percent  error  with  standard  deviation  of  (a)  1,  (b)  le-2,  (c)  le-4,  (d)  le-6,  (e)  le-8  and 
(f)  le-10. 
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The  recovered  indirect  images  with  2x2  resolution  are  not  included  since  these 
tended  to  be  very  accurate  regardless  of  the  error  in  camera  measurement.  This  is  a  result 
of  the  much  lower  condition  numbers  associated  with  the  matrices  used  in  resolving 
images  at  this  resolution  (see  Figure  15).  From  this,  it  is  clear  that  a  higher  desired 
resolution  will  require  less  error  in  the  measurements  recorded  by  the  camera  in  order  to 
maintain  a  recognizable  image.  Additionally,  this  issue  can  be  somewhat  diminished  by 
minimizing  the  condition  number  of  the  matrix  to  be  inverted.  This  can  be  accomplished 
by  varying  the  number,  position  and  layout  of  the  laser  spots,  and  can  be  quickly 
performed  in  simulation  before  beginning  with  an  experimental  setup. 

4.3  Investigative  Questions  Answered 

The  primary  purpose  of  this  work  was  to  determine  if  it  is  possible  to  perform 
indirect  photography  by  using  matrices.  While  this  could  not  previously  be  verified 
experimentally  due  to  flawed  data,  the  simulated  scenarios  provide  strong  evidence  that 
this  can  be  mathematically  accomplished.  The  question  of  the  best  mathematical  method 
for  this  case  was  also  resolved,  in  favor  of  using  the  Moore-Penrose  pseudoinverse  for 
the  required  matrix  inversion. 

Trends  which  determine  the  quality  of  the  recovered  image  as  a  function  of  the 

number,  location  and  pattern  of  the  laser  spots  used  to  illuminate  surface  1  were  sought. 

While  no  indicator  was  found  which  corresponded  to  an  ideal  setup,  it  was  discovered 

that  a  relatively  small  number  of  laser  spots  (less  than  130)  was  required  in  most  cases 

with  this  setup  and  that  continuing  to  add  more  after  approximately  120  never  markedly 

improved  the  final  result.  While  this  will  likely  vary  according  to  the  experimental  setup, 
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this  does  indicate  that  increasing  the  number  of  laser  spots  does  increase  the  recovered 
indirect  image  quality,  but  that  this  cannot  be  continued  indefinitely.  Additionally,  the 
question  of  what  other  methods  could  be  used  to  improve  the  image  was  answered  by 
showing  the  increase  in  MTF  associated  with  moving  the  laser  spots  further  from  the 
field  of  view  on  surface  1/3  and  by  adding  some  variation  to  the  pattern  of  the  laser  spots. 
These  results  are  tempered  by  the  very  high  level  of  accuracy  required  by  the  camera  in 
order  to  produce  recognizable  images. 

4.4  Summary 

By  adjusting  a  combination  of  variables,  it  is  possible  to  resolve  2x2 
checkerboards  at  nearly  any  resolution,  4x4  checkerboards  at  up  to  at  least  32x32 
resolution,  and  8x8  checkerboards  at  8x8  resolution  in  the  given  setup.  As  distances  and 
angles  between  surfaces  change,  it  is  likely  that  these  values  will  change.  However  the 
general  trends  will  remain  and  the  overall  ability  to  indirectly  image  otherwise  hidden 
objects  using  a  matrix  formulation  appears  to  be  mathematically  feasible,  though  it 
requires  very  accurate  equipment. 
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5.  Conclusions  and  Recommendations 


5.1  Chapter  Overview 

This  chapter  contains  the  conclusions  of  this  initial  research  into  matrix-based 
indirect  imaging  techniques,  as  well  as  the  significance  of  these  results.  While  this 
research  was  accomplished  with  specific  geometry  and  hidden  objects,  these  conclusions 
include  several  trends  which  hold  for  matrix-based  indirect  imaging  techniques  in 
general.  Also  included  is  a  brief  overview  of  some  of  the  research  that  has  either  been 
opened  up  by  this  work  or  remains  to  be  done. 

5.2  Conclusions  of  Research 

This  work  has  effectively  demonstrated  that  an  algorithm  using  the  Moore- 
Penrose  pseudoinverse  for  matrix  inversion  can  be  used  to  recover  information  about  a 
hidden  object  using  an  indirect  photography  technique.  Though  this  was  accomplished 
with  the  assumption  that  the  visible  surface  was  Lambertian,  this  process  will  still  work  if 
this  is  not  the  case,  so  long  as  the  BRDF  of  the  surface  is  known.  Additional  conclusions 
which  can  be  drawn  include  that  the  most  important  factor  in  this  case  was  the  pattern 
used  to  determine  the  coordinates  where  the  laser  spots  were  on  the  visible  reflecting 
surface.  Increasing  the  actual  number  of  laser  spots  tended  to  cause  a  quick  initial 
increase  in  resolved  image  quality,  but  it  should  be  noted  that  continuing  to  add  more  did 
not  have  any  positive  influence  after  a  certain  point.  As  a  result,  other  methods,  such  as 
varying  the  layout  of  these  spots,  must  be  pursued  to  increase  image  quality.  This 
becomes  particularly  important  when  the  data  from  the  camera  becomes  noisy  or 
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uncertain,  since  it  is  then  very  important  to  minimize  the  condition  number  of  the  matrix 
to  be  inverted.  It  was  also  determined  that  while  the  modified  MTF  serves  as  an 
excellent  indicator  of  this  image  quality,  it  is  important  to  ensure  that  the  values  used  in 
its  calculation  are  physical,  rather  than  an  unphysical  result  of  mathematical 
manipulation. 

5.3  Significance  of  Research 

While  indirect  photography  has  previously  been  introduced  and  demonstrated, 
this  research  provides  an  alternative  method  which  can  be  pursued.  Using  matrices  and  a 
setup  in  which  the  position  of  the  laser  spot  is  free  to  move  independently  from  the 
camera  FOV  also  increases  the  general  practicality  of  indirect  photography  by  reducing 
the  physical  restraints  on  where  it  can  be  applied.  A  large  unbroken,  uniform  surface  for 
laser  reflection  and  camera  FOV  is  no  longer  required  as  was  previously  the  case.  This 
initial  work  with  the  matrix  formulation  of  the  transport  involved  in  indirect  photography 
also  opens  the  path  for  further  research  into  the  general  form  of  the  matrices  associated 
with  other  experimental  setups,  eventually  allowing  for  indirect  photography  to  be  used 
in  situations  which  do  not  have  a  known  geometry.  Indirect  photography  itself  has  a 
great  deal  of  significance,  particularly  within  the  intelligence  community  and  emergency 
services,  as  a  means  to  identify  and  locate  items  of  importance. 

5.4  Recommendations  for  Future  Research 

There  are  several  questions  raised  by  this  research,  all  of  which  are  deserving  of 
future  research.  One  important  question  which  remained  unanswered  is  what  indicators, 
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if  any,  correspond  to  an  idealized  indirect  photography  setup.  While  this  must  be  a 
function  of  the  matrix  which  is  inverted,  this  research  was  unable  to  find  any  quantity 
which  appropriately  described  the  invertibility  of  this  non-square  matrix.  Finding  such  a 
value  would  greatly  reduce  the  time  needed  for  simulations  and  preparations  once  this 
process  has  reached  a  point  where  it  can  be  applied  for  practical  purposes. 

In  a  similar  vein,  before  indirect  photography  can  be  effectively  used,  it  is 
important  that  further  research  be  done  into  the  general  form  of  the  matrices  involved  in 
indirect  photography.  There  are  several  important  facets  to  this.  While  matrices  were 
considered  for  one  particular  setup  in  the  course  of  this  research,  it  will  be  essential  to 
research  how  these  matrices  change  as  the  geometry  changes  in  order  to  create 
approximate  matrices  for  use  when  the  specific  geometry  is  not  known.  In  conjunction 
with  this,  it  will  also  be  important  to  understand  which  parts  of  the  geometry  are  most 
important  to  accurately  model  or  approximate,  and  how  error  in  those  values  affects  the 
overall  result.  Mathematical  research  should  also  be  conducted  into  which  factors  can 
contribute  to  minimizing  the  condition  number  of  the  matrix  resulting  from  the  geometric 
relations  between  fixed  points  (resolution  elements  on  S2)  and  user-defined  locations 
(laser  points  on  SI). 

Finally,  it  should  be  noted  how  much  of  a  difference  the  layout  and  pattern  of  the 
laser  spots  on  surface  1  can  make.  In  this  paper,  these  were  limited  to  rectangular 
patterns  and  rectangular  patterns  with  small  offsets.  As  there  are  infinitely  many 
different  possible  variations  of  patterns  which  can  be  used,  it  is  recommended  that  a 
method  be  researched  to  determine  the  best  possible  pattern  for  these  laser  spots.  This 
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will  require  that  it  first  be  determined  why  some  patterns  are  able  to  achieve  more 
impressive  results  than  others,  and  then  expand  this  to  maximize  that  effect. 

5.5  Recommendations  for  Action 

Research  into  indirect  photography  remains  in  its  infancy,  and  both  the  theory  and 
application  need  to  be  further  developed  before  it  can  be  put  into  practical  use.  Another 
hindrance  to  currently  applying  these  techniques  is  the  current  state  of  computing  power. 
Until  this  increases,  this  can  only  be  used  for  delayed  information-gathering.  At  this  time, 
the  best  course  of  action  is  for  information  regarding  indirect  photography  to  be 
disseminated  to  various  academic  institutions  for  further  work  and  research,  with  a 
particular  emphasis  on  physics,  mathematics,  and  image  processing. 

5.6  Summary 

Indirect  photography  is  still  in  the  early  stages  of  development  and  much  more 
research  needs  to  be  accomplished  before  it  can  be  used  in  real  world  application. 
However,  this  document  provides  the  basis  for  that  research  to  be  conducted  using  a 
matrix-based  data  reduction  technique,  beginning  with  a  number  of  simplifying 
assumptions  and  slowly  removing  these  until  the  technique  of  indirect  photography 
becomes  practical  and  useful. 
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Appendix  A  -  Representative  Example  of  Simulation  Code 


This  code  is  a  representative  example  of  the  simulation  code  used  to  test  the  effect  of 
varying  different  parameters.  In  this  case,  the  parameter  which  is  varied  is  the  number  of 
laser  spots.  The  user  inputs  the  spatial  frequency  of  the  checkerboard,  the  number  of 
pixels  for  each  dimension  of  the  camera,  the  desired  resolution  and  the  range  of  number 
of  laser  spots  to  consider.  Additionally,  within  the  code  the  offset  from  the  center  point  C 
can  be  modified,  as  well  as  whether  or  not  there  is  stagger.  The  output  of  this  code  gives 
the  object,  a  vector  containing  the  number  of  laserspots,  and  corresponding  vectors  with 
the  modified  MTF  as  well  as  the  condition  number,  and  minimum  eigenvalue  of  the 
matrix  [G12].  Other  similar  code  is  used  to  vary  the  distance  of  laser  spots  lines  from 
point  C,  the  method  of  inversion,  and  the  amount  of  error  in  the  camera  measurement. 
Because  of  the  similarity,  copies  of  this  code  are  not  included  here. 


function  [Object  LaserSpots  MTF  CNum  MinEig] =InImSimB2 (CheckSize , 
CamRow,  CamCol,  Resolution,  MinSpots,  MaxSpots) 

%%  Define  Object  BRDF 
Ob j  ectRows=CamRow*CamCol ; 

Ob j  ectCols= (Resolution) A2 ; 

Object (Ob j  ectRows , Ob j  ectCols ) =0 ; 

Restart=ResolutionA2 / CheckSize ; 
for  i=l : Ob j ectCols 

if  mod ( f loor ( i/ (Restart+ . 001 ) ) , 2 ) ==0 ;  %if  column  corresponds  to 

odd  'row'  of  the  object 
checks 

if  mod ( floor (i/ (Resolution/ CheckSize+ . 001 ) ) , 2 ) ==1 ; 

Object ( : , i ) =1 ; 

end 

else 

if  mod (floor ( (i) / (Resolution/ CheckSize+ . 001 ) )  ,  2 ) ==0 ; 

Object ( : , i) =1 ; 

end 

end 

end 
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Ob j ectAvg=mean (Ob j ect ) ; 

Ob j View (Resolution, Resolution) =0 ;  %Pre-allocate  for  speed 

for  i=l : (Resolution) 

for  j=l: (Resolution) 

Ob j View (i ,  j ) =Ob j  ectAvg ( ( i  —  1 ) * (Resolution) + j  )  ; 

end 

end 

figure ( 1 ) 
colormap (bone) 
imagesc (ObjView) 

%%  Define  S2  positions 
L 1 y= 15.5; 
x2min=3 ; 
x2max=7 ; 

x2step= (x2max-x2min) / (Resolution) ; 

CenterX=5 ; 

%d2  is  measured  from  rear  of  surface2 
d2min=5 .86764; 
d2max=9 .86764; 

d2step= (d2max-d2min) / (Resolution) ; 

x2= ( (x2min+x2step/2 ) :x2step: (x2max-x2step/2 ) ) ; 
d2= (d2min+d2step/2 ) :d2step: (d2max-d2step/2 ) ; 

z2= (d2 ) . / sqrt ( 2 ) ; 
y2=Lly- z2 ; 

CenterY=Lly-7 .8  67 64* sqrt (2)  ; 

%%  Define  S3  positions 
x3min=3 . 25 ; 
x3max=6 . 5 ; 

x3max= (x3max-x3min) +x3min; 
x3step=- (x3max-x3min) / (CamCol-1) ; 

y3top=6 ; 
y3bottom=2 .75; 

y3bottom=y3top- (y3top-y3bottom) ; 
y3step= (y3top-y3bottom) / (CamRow-1) ; 

x3=x3max : x3step : x3min; 
y3=y3bottom : y3step : y3top; 

%%  Create  D23  matrix 

D23 (length (x3) *length (y3) , length (x2) *length (y2) ) =0;  %Preallocate  for 

speed 
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for  h=l : length (y2 ) 

for  i=l : length (x2 ) 

for  j =1 : length (y3 ) 

for  k=l : length (x3 ) 

D23 (k+ ( j-1) ^length (x3) , i+ (h-1) * length (x2) ) = 

( z2 (h) *  sqrt ( (1/2) * ( (y3 ( j ) -y2 (h) ) A2  +  z2 (h) A2)  )  )  / 

(  (  (x3  (k)  -x2  (i)  )  A2  +  (y3  (j  )  -y2  (h)  )  A2  +  z2  (h)  A2)  A  (2)  ) 

end 

end 

end 

end 

for  b=l : MaxSpots-MinSpots  +  1 ; 

LaserSpots (b) =b+MinSpots-l ; 

xlmin=CenterX- 6 / 
xlmax=CenterX+6 ; 

xlstep= (xlmax-xlmin) / (LaserSpots (b) -1) ; 
xlvector=xlmin : xlstep : xlmax; 

ylmin=CenterY- 6 ; 
ylmax=CenterY+6 ; 

ylstep= (ylmax-ylmin) / (LaserSpots (b) -1) ; 
ylvector=ylmax : -ylstep : ylmin; 

xl ( 4  * LaserSpots ) =0 ; 
yl ( 4  * LaserSpots ) =0 ; 

for  i=l : LaserSpots (b) 
if  mod (i ,  3 ) ==0 
Of f set=0 / 

elseif  mod(i,3)==l 
Of f set=-l ; 

else 

Offset=l; 

end 

xl  (i) =xlvector (i)  ; 

xl  ( i+LaserSpots (b) ) =xlmax+Of f set ; 

xl  ( i  +  2 ^LaserSpots (b) ) =xlvector (LaserSpots (b) +l-i) ; 
xl ( i+3*LaserSpots (b) ) =xlmin+Of f set ; 
yl (i) =ylmin+Of f set ; 

yl (i+LaserSpots (b) ) =ylvector (LaserSpots (b) +l-i) ; 
yl (i+2^LaserSpots (b) ) =ylmax+Of f set; 
yl  ( i  +  3^LaserSpots (b) ) =ylvector (i)  ; 

end 

figure ( 2 ) 
plot (xl, yl) ; 
length (xl ) ; 

D12 ( length (x2 ) ^length (y2 ), length (xl )) =0 ;  %Preallocate  for  speed 
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for  i=l : length (xl ) 

for  j=l : length (y2 ) 

for  k=l : length (x2 ) 

Radius (k+ ( j -1) * length (x2  )  ,i)  = 

sqrt  (  (xl  (i)  -x2  (k)  )A2+(yl(i)-y2  (j)  )A2  +  z2  (j)A2)  ; 
CosS (k+ ( j -1 ) * length (x2 ) , i) = 

z2 ( j ) . /Radius (k+ ( j -1 ) * length (x2 ) , i ) ; 

CosD ( k+ ( j - 1 ) * length (x2 ) , i ) = 

sqrt (1/2) * (yl (i) -y2 (j)+z2(j))./ 

Radius (k+ ( j -1 ) * length (x2 )  ,  i )  ; 

D12 (k+ ( j-1) ^length (x2) , i) = 

CosS (k+ (j-1) * length (x2) ,  i ) . * 

CosD (k+ ( j -1) * length (x2) , i ) . / (Radius (k+ ( j-1) * 
length (x2 ) , i ) . A2 ) ; 

D12 (k+ ( j-1) * length (x2) , i) = 

(z2(j)*sqrt((l/2)*((yl(i)-y2(j))A2  +  z2(j)A2)))/ 

( ( (xl (i) -x2 (k) ) A2+(yl (i)-y2 (j) ) A2+z2 (j) A2) A (2) ) 

end 

end 

end 

%%  Cleanup  unneeded  variables 
clear  ObjView  ObjectAvg 

clear  Alpha  Gammal  Gamma2  GammaStepB  GammaStepT  GammaVB  GammaVT 
clear  LGV  LGVB  LPV  Phil  Phi2  PhiStep  PhiV 

clear  Sigma  XOrigin  YOrigin  dmO  h  i  j  k  s  v  w  Lly  L2k  d2min  d2max 
clear  d2step  x2min  x2max  x2step  x3step  y3step 
clear  x3  y3  CamCol  CamRow 

%%  Calculate  Intensity 
Intensity^ (Ob j  ect . *D2  3 ) *D12  ; 

%%  Reverse  Calculate  BRDF  of  Object 
BRDF= (pinv (D12 ’ ) * Intensity ' ) ’ . /D23 / 

for  i=l : length (BRDF ( 1 ,:)  ) 

BRDFAvg ( i ) =sum (BRDF (:,i)) /length (BRDF ( : , 1) ) ; 

end 

minBRDF=min (BRDFAvg) ; 
maxBRDF=max (BRDFAvg) ; 

BRDFAvg= (BRDFAvg-minBRDF) / (maxBRDF-minBRDF) ; 

for  i=l : length (x2 ) 

for  j=l : length (d2 ) 

BRDFMat (i ,  j ) =BRDFAvg ( ( i — 1 ) * length (d2 ) + j ) ; 

end 

end 

%f igure ( 3 ) 

%colormap (bone) 
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%imagesc (BRDFMat) 


DarkSum=0 ; 

for  i=l : Ob j ectCols 

if  mod ( f loor ( i/ (Restart+ . 001 ) ) , 2 ) ==0 ;  %if  column  corresponds  to 

odd  'row'  of  the  object 
checks 

if  mod ( floor (i/ (Resolution/ CheckSize+ . 001 ) ) ,  2 )  ==1 / 
DarkSum=DarkSum+BRDFAvg ( : , i) ; 

end 

else 

if  mod ( floor (( i ) / ( Re s olut ion/ Checks ize+ . 001 ) ) , 2 ) ==0 ; 
DarkSum=DarkSum+BRDFAvg ( :  ,  i)  ; 

end 

end 

end 

LightSum=sum ( sum (BRDFMat) ) -DarkSum; 

MTF (b) = (DarkSum- Light Sum) / ( sum ( sum (BRDFMat) ) ) ; 

CNum (b) =norm (D12 ) *norm (pinv (D12 ) ) ; 

MinEig (b) =eigs (D12*D12 ' , 1 ,  ' sm ’ ) ; 

clear  Intensity  BRDF  D12  Radius  CosS  CosD 
end 

figure ( 1 ) 

plot (Laser Spots*  4 , MTF) 
set (gca,  ' f ontsize  '  , 12 ) 

xlabel (' Number  of  laser  spots f ontsize 15 ) 
ylabel ( ' Modified  MTF ' ,  1 f ontsize ' , 15 ) 

figure  (2 ) 

plot (Laser Spots* 4 , CNum) 
set (gca, 1 f ontsize 1 , 12 ) 

ylabel (’ Condition  number  of  inverted  matrix f ontsize ! , 15 ) 
xlabel (’ Number  of  laser  spots  1 ,  1 f ontsize 1 , 15) 

figure  ( 3 ) 

plot (LaserSpots* 4 , MinEig) 
set (gca,  1 f ontsize  1  , 12 ) 

ylabel (' Minimum  eigenvalue  of  B*B ' ’ f ontsize ', 15 ) 
xlabel (' Number  of  laser  spots  1 ,  1 f ontsize 1 , 15) 

end 
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14.  ABSTRACT 

Indirect  photography  is  a  recently  demonstrated  technique  which  expands  on  the  principles  of  dual 
photography  and  allows  for  the  imaging  of  hidden  objects.  A  camera  and  light  source  are  collocated  with 
neither  having  line-of-sight  access  to  the  hidden  object.  Light  from  the  source,  a  laser,  is  reflected  off  a 
visible  non-specular  surface  onto  the  hidden  object,  where  it  is  reflected  back  to  the  initial  non-specular 
surface  and  collected  by  the  camera.  This  process  may  be  repeated  numerous  times  for  various  laser 
spot  positions  to  yield  slightly  different  camera  images  due  to  a  variation  in  the  illumination  of  the  object. 
These  images  can  then  be  used  to  construct  an  “indirect”  image  of  the  hidden  object.  This  work  provides 
an  alternative  method  of  processing  the  camera  images  by  modeling  this  system  as  a  set  of  transport  and 
reflectance  matrices.  This  approach  reduces  the  required  size  of  the  visible  scattering  surface.  Matrix 
formulation  and  those  parameters  shown  in  simulation  to  improve  indirect  image  quality  as  measured  by 
a  modified  MTF,  including  the  method  of  matrix  inversion,  number  and  pattern  of  laser  spots,  are 
discussed. 
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